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I. INTRODUCTION 



The Graphics and Video Laboratory of the Department of Computer Science at the 
Naval Postgraduate School permits the researcher to create three-dimensional visual 
simulations from digital terrain data [Ref. 1]. Specialized graphics hardware allows the 
display of such simulations in near-real time. The goal of a good part of the work in the 
lab is the creation of a movie-like view of movement over and on terrain, with 
increasingly complex movement animation models. Such projects have strained the 
equipment’s capabilities. One method of increasing available computing power is to 
harness multiple heterogeneous machines together in some distributed computing 
organization. It requires communication between the various machines, as well as 
carefully matching each machine’s capabilities to its assigned tasks. 

A. PROBLEM 

Rapid turnover of inexperienced students at the Naval Postgraduate School makes 
the creation of complex simulations difficult to manage. The learning curve becomes 
steeper as the lab’s capabilities increase. One of the areas of difficulty has been inter- 
computer communications. So much time has been spent on designing, coding, and 
debugging communication software, little has been left for the original research. We set 
out to provide an easy-to-use, yet powerful, set of tools to aid in the development of 
multi-computer projects. 

1. Approach 

A communication protocol can be optimized for large data transfers, or small 
data transfers, or both. Efforts to optimize for both are both complex and difficult 
[Refs. 2, 3]. File transfer protocols such as FTP in the Defense Advanced Research 
Project Agency (DARPA) Internet domain and uucp in the UNIX domain can be used for 



1 



large data transfers. Their overhead 1 is high. This overhead cannot be tolerated in a 

real-time problem 2 . Our visual simulation efforts rely on small data transfers to 
communicate among machines. These small messages are typically commands and 
changing status indicators. Transferring the entire “world view” is only a reasonable task 
during initialization or reset. Hence, we designed our protocols for small messages. 

2. Design Criteria 

The design criteria for developed protocols were simplicity, ease of use, 
portability, and efficiency. Rapid turnover of inexperienced students at the Naval 
Postgraduate School makes simplicity of paramount importance. Inevitably, changes 
will be required and only a simple protocol is easily modified to take advantage of new 
capabilities. Much the same argument, and generally good software design practice, 
make ease of use only slightly less important. Almost all operating system-level aspects 
are hidden from the application. The number of other machines to be connected to, a use 
of dynamic memory allocation, and the names of the other machines are the only 
concerns for the application setting up a connection. The synchronization, or lack 
thereof, in communication between machines is a design decision. 

Portability dictated our use of TCP/IP, an integral part of the Defense Data 
Network (DDN). Efficient use of processor power was considered more important than 
efficient use of the network resources. The network is shared by the entire Computer 
Science Department, but is not heavily loaded. 



1 The cost of creating a file and then spawning a process to send it is high. On the receiving end, there is the cost 
of creating the file and then reading it. Even a zero-cost file transfer protocol will require all this overhead. 

2 Large data transfers, in real-time systems, will not be possible until 100 MByte/Sec networks are commonly 
available. 
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B. BACKGROUND 



1. Visual Simulation 

a. Vision and Information Presentation 

The eye has the largest bandwidth of any human sensory organ. Proper 
use of this capability is a challenge to all scientists. Static graphs are used in most 
disciplines to show the relationships between a limited number of variables. These two- 
dimensional representations convey information more readily to human beings than 
would a table of the underlying numbers. [Ref. 4: pp. 8-12] 

Time, a common independent variable, is often one dimension on a graph. 
The other dimension is a single dependent variable. To portray additional variables in 
one presentation is a frequently occurring requirement. Various techniques such as 
multiple colored lines, multiple icons, and perspective drawing are used. With each 
technique, only a few additional variables are added before the graph becomes 
incomprehensible . 

Pictures, particularly those in color, have a dense information content. 
Unless blind, we live in a world of pictures. Human beings can recognize many 
differences between two similar pictures. One presentation portrays many different 
variables. When a series of pictures are presented, the time variable is easily correlated 
to the actual time of presentation. When a series of pictures is presented rapidly, the 
experience approaches reality, partly explaining the success of moving pictures and 
television. 

Animation creates visual images with an explicit time dimension, in 
addition to two or three spatial dimensions. Using actual time to portray the 
experimental time variable allows at least one more dependent variable on the display. 
Images can be as simple as a changing graph, or as complex as a feature-length cartoon. 
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However, animation creates its effect with the playback of prerecorded scenes [Ref. 5]. 
It is not suitable for providing immediate feedback to a researcher. 

b. Definition 

Visual simulation is the creation, by computer, of a realistic, easily- 
modified, moving image from the mathematical model of a phenomenon. Realism 
implies high-resolution, color graphics. Movement implies adequate floating point 
calculation capacity to recalculate the model and its graphical representation between 
display refresh cycles. Easy modification implies a well-designed computer application. 

Visual simulation allows a researcher to experiment easily with his 
subject. Ideally, we display a realistic approximation of part of the world. The 
experimenter then manipulates some part of this visual simulation and receives 
immediate visual feedback. The rapidly refreshed display is one key to visual realism. 
Such a display allows the direct manipulation of the visual simulation, making it easy 
and intuitive to use [Ref. 6]. Ease of use allows the researcher to concentrate on the 
research question, not the display methodology or the computer interface. 

c. Examples 

Recent visual simulation projects of the Graphics and Video Laboratory 
include speed control of autonomous vehicles [Ref. 7], control of autonomous walking 
machines [Ref. 8], rule-based control of autonomous underwater vehicles [Ref. 9], 
interactive moving platforms [Ref. 10] and combat vehicle control [Ref. 11]. Each of 
these projects exceeded the capacity of a single workstation. The speed control and 
interactive moving platform projects, written entirely in C, used two Silicon Graphics, 
Inc. IRIS workstations, allowing multiple simultaneous views. The other projects all 
required a rule-based artificial intelligence component, best programmed in Lisp for ease 
of modification. Running the Lisp subsystem on the IRIS workstation gave an 
unacceptably low refresh rate and correspondingly poor realism [Ref. 12]. Placing the 



4 



Lisp subsystem on another machine improved the refresh rate of the IRIS workstation 
used for the graphics display. 

2. Computer System Architecture 

Computer systems can have a distributed or a non-distributed architecture. 
Distributed architectures have only one characteristic in common, more than one 
processor used to accomplish the task. Beyond this, many different approaches have 
been tried [Ref. 13]. Identical processors give a homogeneous architecture. Different 
processors give a heterogeneous architecture. Either distributed architecture may 
incorporate shared memory or it may not. The separate processors can be closely or 
loosely coupled. Communication between processors can be via shared memory, 
common bus, or some form of communications network. Communication via some 
combination of the above, such as a file server on a local area network, is also 
common [Ref. 3]. In the Computer Science Department at the Naval Postgraduate 
School, a heterogeneous mix of stand-alone workstations, file server supported 
workstation clusters, and minicomputers communicates via Ethernet. 

Programming distributed architectures has inspired creativity. The 
fundamental problems with distributed programming are the communications between 
processes and the temporal interaction of the processes. Communicating sequential 
processes [Ref. 14], distributed processes [Ref. 15], and remote procedure calls 
[Refs. 2, 16] have all been proposed as primitives to hide message passing from the 
programmer. Remote procedure calls [Refs. 2, 3] and communicating sequential 
processes [Ref. 17] have been implemented. However, even today, none of these is 
generally available as a standard mechanism across varied architectures. We have 
created simpler (but less general) communication routines for use among heterogeneous, 
distributed, standalone computers. 
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Complex projects can require the resources of more than one computer. 
Graphics portions are best handled by the specialized hardware of a graphics workstation, 
such as a Silicon Graphics, Inc. ERIS. Artificial intelligence portions are best handled by 
a Lisp machine, such as a Symbolics* or a Texas Instruments Explorer**. Database 
requests can be made to a machine with appropriate database software. A general 
purpose computer, such as the Digital Equipment Corporation VAX***, can be used for 
additional processing power, file storage, or other administrative support. Providing easy 
access across such a mix of heterogeneous computers is a large task [Ref. 3], The simple 
mechanism described in this work gives communication access between cooperating 
processes running on diverse hardware. It leaves temporal design to the application 
developer, while providing the tools for synchronous and asynchronous interaction. 

3. Communication 

Communications between computers cooperating on a task can be one-to-one, 
many-to-one, or one-to-many. It can be synchronous or asynchronous. Any, or all, of 
these can be required for one visual simulation. 

One-to-one, or direct connect, communications puts the lowest load on the 
network when there are few messages to be sent. A single virtual channel between the 
two processes is required. Each communication between any two processes comprises 
one message. All messages are known to be intended for the receiving process. These 
messages can be sent synchronously or asynchronously. Direct connect communication 
requires one action by the sender and one by the receiver. With more processors, 



* Symbolics is a trademark of Symbolics, Incorporated. 

** Explorer is a trademark of Texas Instruments Incorporated. 

*** VAX is a registered trademark of Digital Equipment Corporation 
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potential virtual channels grow in number geometrically. For a fully connected network, 
the virtual channels required can exceed capacity. The potential messages required also 
grow geometrically in number. 

One-to-many, or broadcast, communications puts the lowest load on die 
sending process. A message is sent to all other processes that are connected to it. It 
requires one action by the sender, and two actions by each receiver (the reception and a 
decision on whether the message is intended for that receiver). It also places one to n 
messages on the network (depending on how the network and the broadcast protocols are 
designed). It is primarily used in an asynchronous mode, although synchronous protocols 
could be designed. 

Many-to-one communications puts the highest load on the receiving process. It 
requires two actions by the receiver on every message that is sent by any connected 
process. It is also a primarily asynchronous method. The receiver portion of a process 
sees many-to-one whenever broadcast protocols are the only ones used in a visual 

simulation. 

C. Organization 

The previous sections of this chapter provide background on visual simulation, 
distributed architectures, and communication paradigms. Chapter II describes the 
hardware and software environment in the Computer Science Department at the Naval 
Postgraduate School. The protocols developed are discussed in Chapter III. Chapter IV 
describes the implementation of the protocols. Chapter V covers the use of these 
protocols. The performance of the protocols is detailed in Chapter VI. Chapter VII 
concludes with a discussion of limitations, future extensions and research topics, and 
summarizes the research conducted. Listings of the program source code for each of the 
hardware systems are included as Appendices. 
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